The Sparse PCA Problem: Optimality Conditions and Algorithms

Sparse principal component analysis (PCA) addresses the problem of finding a linear combination of the variables in a given data set with a sparse coefficients vector that maximizes the variability of the data. This model enhances the ability to interpret the principal components, and is applicable in a wide variety of fields including genetics and finance, just to name a few. We suggest a necessary coordinate-wise-based optimality condition, and show its superiority over the stationarity-based condition that is commonly used in the literature, and which is the basis for many of the algorithms designed to solve the problem. We devise algorithms that are based on the new optimality condition, and provide numerical experiments that support our assertion that algorithms which are guaranteed to converge to stronger optimality condition, perform better than algorithms that converge to points satisfying weaker optimality conditions. Principal component analysis (PCA) is a well known data-analytic technique that linearly transforms a given set of data to some equivalent representation. This transformation is defined in such a manner that any variable in the new representation, called a principal component (PC), expresses most of the variance in the data which is not expressed by the PCs that preceded it. The linear combination defining each of the PCs is given by a coefficients (also termed loadings) vector. In terms of the covariance (or correlation) matrix of the data, the coefficients vector of the k-th PC is the eigenvector that corresponds to the k-th largest eigenvalue [9]. One major drawback of PCA is that commonly the coefficients vectors are dense, i.e. each PC is a linear combination of much, if not most, of the original variables, which causes a difficulty in interpreting the obtained PCs. This disadvantage encouraged a wide interest in the sparsity constrained version of PCA, which imposes an additional constraint, enforcing the coefficients vector not to exceed some predetermined